Method and System for Registering a Patient with a 3D Image Using a Robot

ABSTRACT

A method of operating a medical registration system, the medical registration system comprising a robot which carries a surface point sampling device and comprising a computer connected to the robot, to obtain a registration of a 3D image of a patient with the patient (P) by: a) acquiring the 3D image of the patient; b) acquiring an initial registration of the 3D image with the patient as the registration; c) instructing the robot to sample the spatial locations of N different points on the surface of the patient using the surface point sampling device, wherein N is a positive integer; d) updating the registration based on the spatial locations of the N sampled points; and e) repeating steps c) and d) until a predetermined exit condition is fulfilled.

TECHNICAL FIELD

The present invention relates to a computer implemented method forobtaining a registration of a 3D image of a patient with the patient andto a corresponding computer program and system.

SUMMARY

Many medical applications require a registration of a patient with a 3Dimage of the patient. The registration in this context does for examplemean to adapt the 3D image so as to correspond to the surface of thepatient. The 3D image of the patient thus represents at least a part ofthe shape of the patient, that is the surface or outer contour of thepatient. The 3D image can for example be a 3D array of voxels whichrepresent the surface of the patient or a set of locations of pointswhich represent the surface of the patient.

In general, the 3D image is obtained at a first point in time, forexample before a treatment of the patient. The registration of thepatient with the 3D image of the patient typically happens at a secondpoint in time later than the first point in time, for example during orimmediately before a treatment of the patient. The registrationtypically involves obtaining a set of samples representing the spatiallocations of points on the surface of the patient and matching the 3Dimage to the set of samples, for example by rigid or elastic fusion.

In general, surface registration is a cumbersome procedure for the userbecause it takes quite some time to acquire enough and suitable pointsfor surface matching. In addition, the quality of the sampled points isoften sub-optimal because not enough points are sampled or points aresampled at locations which are not contributing to the improvement ofthe registration.

The purpose of the present invention is to improve the registrationprocess by utilizing a robot.

The method, the program and the system are defined by the appendedindependent claims. Advantages, advantageous features, advantageousembodiments and advantageous aspects of the present invention aredisclosed in the following and contained in the subject-matter of thedependent claims. Different advantageous features can be combined inaccordance with the invention wherever technically expedient andfeasible. Specifically, a feature of one embodiment which has the sameor a similar function to another feature of another embodiment can beexchanged with said other feature, and a feature of one embodiment whichadds an additional function to another embodiment can in particular beadded to said other embodiment.

The present invention relates to a method of operating a medicalregistration system, the medical registration system comprising a robotwhich carries a surface point sampling device and comprising a computerconnected to the robot, to obtain a registration of a 3D image of apatient with the patient.

The method involves a first step of acquiring the 3D image of thepatient. As explained above, the 3D image represents at least a part ofthe surface of the patient, for example as a 3D array of voxels or a setof locations of points on the surface.

The method further involves a second step of acquiring an initialregistration of the 3D image with the patient as the registration. Theinitial registration can be calculated by any suitable approach, such asan approach using landmarks, a new 3D image of the patient, for examplecaptured using the time-of-flight camera or a stereoscopic camera, or a(small) set of sampled points on the surface of the patient.

The method further involves a third step of instructing the robot tosample the spatial locations of N different points on the surface of thepatient using the surface sampling device, wherein N is a positiveinteger. This means that N is 1, 2, 3 or more. The result of this thirdstep is a set of N sample points, which means a set of N spatiallocations.

The method further involves a fourth step of updating the registrationbased on the spatial locations of the N sampled points, which are the Nsampled points which were sampled in the third step. Updating theregistration can mean the calculation of a completely new registrationbased on the N sampled points sampled in the third step and any sampledpoints sampled previously, for example in a previous iteration of themethod, or an amendment of the registration considering the N sampledpoints.

The method further involves a fifth step of repeating the third step ofinstructing the robot to sample the spatial locations of N differentpoints and the fourth step of updating the registration until apredetermined exit condition is fulfilled. This means that the presentmethod iterates through the third and fourth steps at least twice ormore. It shall be noted that the number N of sampled points can vary inevery iteration.

The iterative approach according to the present invention allows toevaluate the registration after each iteration and to select the surfacepoint to be sampled in the next iteration accordingly, for example inorder to increase the accuracy and/or reliability of the registration.Since there is the initial registration or the registration after acompleted iteration of the method, an approximate topography of thepatient's surface is known, such that the approximate spatial locationsof the points to be sampled are known. A surface point can then beexpected at or near the approximate spatial location. The exact spatiallocation can then be sampled.

There are many approaches for sampling spatial locations of points onthe surface of a patient. They can basically be divided intocontact-based and contactless approaches. In a contact-based approach, aprobe approaches the surface of the patient until it is in contact andthe location of the point of the probe which contacts the surface isdetermined as the spatial location of the sampled point. The probe isapproached towards the surface by the robot. The spatial location of theprobe can be obtained from the pose of the robot, a marker deviceattached to the probe or to the robot or any other suitable technique.The contact of the probe and the surface can for example be determinedby a sensor in the probe or a sensor in the robot. The sensor is forexample a pressure sensor.

One contactless approach involves a light beam such as a laser beam,which is generated by a light beam generator and directed towards thesurface of the patient. In one example, the spatial location at whichthe light beam hits the surface of the patient is determined from astereoscopic image which shows the light beam on the surface of thepatient. Such a system is known as Z-Touch® of the applicant. In such asystem, a marker can optionally be attached to the surface pointsampling device such that it can be tracked.

In another example, the light beam source carries a marker device andcomprises a range finder which determines the distance between the lightbeam generator and the light spot on the surface of the patient. Thespatial location of the sampled point is then determined from thespatial position of the marker device attached to the light beam source,the orientation of the light beam relative to the marker device and thedistance between the light beam source and the light spot on the surfaceof the patient.

In one embodiment, the exit condition of the fifth step is at least oneof a time lapse, the total number of sampled surface points, theaccuracy of the registration or the registration confidence. A timelapse means that a predetermined amount of time is budgeted for theregistration process, in particular for the sampling process. Once thebudgeted amount of time is consumed, the workflow ends. This means forexample that the current iteration continues until the N point aresampled, but no new iteration is started.

The total number of sampled surface points means the number of sampledsurface points over all iterations of the method.

The accuracy of the registration describes how accurately the sampledpoints match the registered 3D image. One possible metric fordetermining the accuracy of the registration is a root mean square ofthe distances of the sampled points to the registered 3D image. Thedistance of a sampled point to the surface as represented by theregistered 3D image is for example the shortest possible distance of thesampled point to the 3D surface. The surface as represented by theregistered 3D image is also referred to as registered 3D surface.

The registration confidence for example identifies the probability thatthe found registration is the best possible registration. Theregistration confidence can have at least the two following aspects.Those aspects are based on the fact that different registrations havedifferent accuracies, for example different root mean squares of thedistances between the sampled points and the registered 3D surface asexplained above. This means that a plurality of registrations results inan error function which associates a registration with an accuracy. Thebest possible registration is the one corresponding to the globalminimum of the error function.

In one aspect of the registration confidence, the relation between theaccuracy of the best registration, that is the one with the globalminimum of the error function, and the accuracy of the second bestregistration, that is the registration with the lowest local minimum ofthe error function which is not the global minimum, fulfills apredetermined criterion, such as a predetermined relative or absolutedifference.

In another aspect, the error function of the registration shall have asharp minimum, which means that the accuracy of a registration close tothe best registration is significantly lower than the accuracy of thebest registration.

In one embodiment, sampling the spatial location of a surface point inthe third step involves moving, by the robot, the surface point samplingdevice towards the surface of the patient until it is in contacttherewith as explained above. This is performed for each of the Ndifferent points. The surface point sampling device is for example movedperpendicularly towards the surface at the point to be sampled, whereinthe orientation of the surface at the point to be sampled is determinedfrom the registered 3D image, i.e. the approximate topography.

In one embodiment, the third step of instructing the robot to sample thespatial locations of N different points involves, for each of the Ndifferent points, instructing the robot to move the surface pointsampling device into a sampling position and instructing the surfacepoint sampling device to sample a surface point. This relates to acontactless technique as explained above. In this embodiment, sampling asurface point for example involves determining the distance from thesampled surface point to the surface point sampling device andcalculating the spatial location of the surface point from said distanceand the sampling position of the surface point sampling device asexplained above. In another example, this involves creating a light spoton the surface of the patient and determining the spatial location ofthe light spot as the spatial location of the surface point.

Within each iteration, the robot consecutively moves the surface pointsampling device into N sampling positions. It is not necessary to stopthe surface point sampling device at the sampling position for thesampling operation. It is also possible to sample a surface point duringa motion of the surface point sampling device, but when it is at or nearthe corresponding sampling position. However, the robot can becontrolled such that the speed of the surface point sampling device at asampling position is below a predetermined threshold. This improves theaccuracy of the sampled spatial location of the sampled point. However,it is of course possible to stop the surface point sampling device at asampling position for sampling a surface point.

In one embodiment, the sampling position is a position in which at leastone of the distance or the orientation between the surface pointsampling device and the surface of the patient is within a predeterminedrange. This leads to a shape and/or size of the light spot on thepatient's surface which is easily and/or exactly recognizable, forexample using a stereoscopic camera.

In one embodiment, the sampling position is a position in which a markerdevice attached to the robot or the surface point sampling device iswithin a predetermined range relative to a medical tracking system, forexample relative to a stereoscopic camera of the medical trackingsystem. This means that the position of the marker device can beascertained by the medical tracking system, either at all or with acertain accuracy.

An advantage of using a robot for sampling surface points is thatsurface points can be sampled even if the marker device is not withinthe field-of-view of the medical tracking system. In this case, theposition of the surface point sampling device, when a surface point issampled, can be determined from the last position of the surface pointsampling device measured by the medical tracking system and the movementof the robot since then. This applies to both contact-based andcontactless techniques.

In one embodiment, the sampling position depends on the surface point tobe sampled. As explained above, the approximate topography of thepatient's surface is known from the latest registration of the 3D image.This means that the approximate location of a surface point to besampled is known. In this embodiment, the sampling position ispreferably optimized for the approximate spatial location of the pointto be sampled.

An important aspect of the present invention relates to planning thesurface points to be sampled in an iteration of the method. Theiterative approach of the present invention allows for an optimizedplanning, for example in teams of the total number of surface points tobe sampled for a reliable fusion of the 3D image, and therefore theamount of time required for the sampling process.

In one embodiment, a surface point to be sampled is determined based onthe registered 3D image or a registered atlas. The registered 3D imageor the registered atlas represents an approximate topology of thepatient's surface, such that the complexity of particular areas of thepatient's surface can be determined from the registered 3D image or theregistered atlas and the number of surface points to be sampled in acertain area of the patient's surface can for example correlate with thecomplexity of this area. The more complex the area is, the higherbecomes the number of points to be sampled in this area.

In one embodiment, a surface point to be sampled is determined based ona registered atlas and the atlas defines the number of points to besampled in a predefined area of the surface. This means that the numberof points to be sampled for a particular area is not calculated on thefly, but is pre-stored in the atlas or along with the atlas.

In one embodiment, a surface point to be sampled is a point in an areaof the surface with a high curvature or comprising a saddle point. Ahigh curvature means a curvature above a predetermined threshold.

In another embodiment, a surface point to be sampled is a point with amaximum distance to the other surface points sampled so far, which meansall surface points sampled so far or the surface points sampled so farin the current iteration of the method. This embodiment assures that alarge area of the patient's surface is sampled with a small number ofsampled points.

In one embodiment, a surface point to be sampled is determined based ona co-registered optical image of the patient. This means that an opticalimage of the patient is acquired. The optical image of the patient canbe a 2D image or a 3D image. The optical image is for example capturedby a video camera. The optical image being co-registered means that theviewing direction of the optical image relative to the patient and/orthe registered 3D image is determined.

In this embodiment, it is for example possible to identify parts of theco-registered optical image which do not show the surface of thepatient. It is for example possible to identify areas of the surfacewhich are covered by objects such as draping, tubing etc. It does notmake sense to sample a point on the surface of such an object, such thatfor example areas not to be sampled can be determined from theco-registered optical image.

In one embodiment, the surface points to be sampled are selected suchthat the robot remains in one or more safe working areas. A safe workingarea is a spatial area in which the robot can move without the risk ofcolliding with an object or a person. Safe working areas are for examplepredefined and stored in the computer. In an optional addition to thisembodiment, the computer controls the robot such that it remains in oneor more safe working areas while it moves the surface point samplingdevice from one sampling position into another sampling position.

In one embodiment, the third step involves trajectory planning of thesurface point sampling device. Trajectory planning for example optimizesthe order in which the N surface points are sampled such that the timerequired for moving the surface point sampling device into all Nsampling positions and/or the length of the trajectory are minimized.

The present invention further relates to a program which, when runningon a computer, causes the computer to perform the method as explainedabove and/or to program a storage medium on which the program is stored,in particular in a non-transitory form.

The present invention further relates to a computer on which theaforementioned program is stored and/or run and to a medicalregistration system comprising the aforementioned computer, a robot anda surface point sampling device attached to the robot.

Definitions

The method in accordance with the invention is for example a computerimplemented method. For example, all the steps or merely some of thesteps (i.e. less than the total number of steps) of the method inaccordance with the invention can be executed by a computer (forexample, at least one computer). An embodiment of the computerimplemented method is a use of the computer for performing a dataprocessing method. An embodiment of the computer implemented method is amethod concerning the operation of the computer such that the computeris operated to perform one, more or all steps of the method.

The computer for example comprises at least one processor and forexample at least one memory in order to (technically) process the data,for example electronically and/or optically. The processor being forexample made of a substance or composition which is a semiconductor, forexample at least partly n- and/or p-doped semiconductor, for example atleast one of II-, III-, IV-, V-, VI-semiconductor material, for example(doped) silicon and/or gallium arsenide. The calculating steps describedare for example performed by a computer. Determining steps orcalculating steps are for example steps of determining data within theframework of the technical method, for example within the framework of aprogram. A computer is for example any kind of data processing device,for example electronic data processing device. A computer can be adevice which is generally thought of as such, for example desktop PCs,notebooks, netbooks, etc., but can also be any programmable apparatus,such as for example a mobile phone or an embedded processor. A computercan for example comprise a system (network) of “sub-computers”, whereineach sub-computer represents a computer in its own right. The term“computer” includes a cloud computer, for example a cloud server. Theterm “cloud computer” includes a cloud computer system which for examplecomprises a system of at least one cloud computer and for example aplurality of operatively interconnected cloud computers such as a serverfarm. Such a cloud computer is preferably connected to a wide areanetwork such as the world wide web (WWW) and located in a so-calledcloud of computers which are all connected to the world wide web. Suchan infrastructure is used for “cloud computing”, which describescomputation, software, data access and storage services which do notrequire the end user to know the physical location and/or configurationof the computer delivering a specific service. For example, the term“cloud” is used in this respect as a metaphor for the Internet (worldwide web). For example, the cloud provides computing infrastructure as aservice (IaaS). The cloud computer can function as a virtual host for anoperating system and/or data processing application which is used toexecute the method of the invention. The cloud computer is for examplean elastic compute cloud (EC2) as provided by Amazon Web Services™. Acomputer for example comprises interfaces in order to receive or outputdata and/or perform an analogue-to-digital conversion. The data are forexample data which represent physical properties and/or which aregenerated from technical signals. The technical signals are for examplegenerated by means of (technical) detection devices (such as for exampledevices for detecting marker devices) and/or (technical) analyticaldevices (such as for example devices for performing imaging methods),wherein the technical signals are for example electrical or opticalsignals. The technical signals for example represent the data receivedor outputted by the computer. The computer is preferably operativelycoupled to a display device which allows information outputted by thecomputer to be displayed, for example to a user. One example of adisplay device is an augmented reality device (also referred to asaugmented reality glasses) which can be used as “goggles” fornavigating. A specific example of such augmented reality glasses isGoogle Glass (a trademark of Google, Inc.). An augmented reality devicecan be used both to input information into the computer by userinteraction and to display information outputted by the computer.Another example of a display device would be a standard computer monitorcomprising for example a liquid crystal display operatively coupled tothe computer for receiving display control data from the computer forgenerating signals used to display image information content on thedisplay device. A specific embodiment of such a computer monitor is adigital lightbox. The monitor may also be the monitor of a portable, forexample handheld, device such as a smart phone or personal digitalassistant or digital media player.

The expression “acquiring data” for example encompasses (within theframework of a computer implemented method) the scenario in which thedata are determined by the computer implemented method or program.Determining data for example encompasses measuring physical quantitiesand transforming the measured values into data, for example digitaldata, and/or computing the data by means of a computer and for examplewithin the framework of the method in accordance with the invention. Themeaning of “acquiring data” also for example encompasses the scenario inwhich the data are received or retrieved by the computer implementedmethod or program, for example from another program, a previous methodstep or a data storage medium, for example for further processing by thecomputer implemented method or program. The expression “acquiring data”can therefore also for example mean waiting to receive data and/orreceiving the data. The received data can for example be inputted via aninterface. The expression “acquiring data” can also mean that thecomputer implemented method or program performs steps in order to(actively) receive or retrieve the data from a data source, for instancea data storage medium (such as for example a ROM, RAM, database, harddrive, etc.), or via the interface (for instance, from another computeror a network). The data acquired by the disclosed method or device,respectively, may be acquired from a database located in a data storagedevice which is operably to a computer for data transfer between thedatabase and the computer, for example from the database to thecomputer. The computer acquires the data for use as an input for stepsof determining data. The determined data can be output again to the sameor another database to be stored for later use. The database or databaseused for implementing the disclosed method can be located on networkdata storage device or a network server (for example, a cloud datastorage device or a cloud server) or a local data storage device (suchas a mass storage device operably connected to at least one computerexecuting the disclosed method). The data can be made “ready for use” byperforming an additional step before the acquiring step. In accordancewith this additional step, the data are generated in order to beacquired. The data are for example detected or captured (for example byan analytical device). Alternatively or additionally, the data areinputted in accordance with the additional step, for instance viainterfaces. The data generated can for example be inputted (for instanceinto the computer). In accordance with the additional step (whichprecedes the acquiring step), the data can also be provided byperforming the additional step of storing the data in a data storagemedium (such as for example a ROM, RAM, CD and/or hard drive), such thatthey are ready for use within the framework of the method or program inaccordance with the invention. The step of “acquiring data” cantherefore also involve commanding a device to obtain and/or provide thedata to be acquired. In particular, the acquiring step does not involvean invasive step which would represent a substantial physicalinterference with the body, requiring professional medical expertise tobe carried out and entailing a substantial health risk even when carriedout with the required professional care and expertise. In particular,the step of acquiring data, for example determining data, does notinvolve a surgical step and in particular does not involve a step oftreating a human or animal body using surgery or therapy. In order todistinguish the different data used by the present method, the data aredenoted (i.e. referred to) as “XY data” and the like and are defined interms of the information which they describe, which is then preferablyreferred to as “XY information” and the like.

The invention also relates to a program which, when running on acomputer, causes the computer to perform one or more or all of themethod steps described herein and/or to a program storage medium onwhich the program is stored (in particular in a non-transitory form)and/or to a computer comprising said program storage medium and/or to a(physical, for example electrical, for example technically generated)signal wave, for example a digital signal wave, carrying informationwhich represents the program, for example the aforementioned program,which for example comprises code means which are adapted to perform anyor all of the method steps described herein.

Within the framework of the invention, computer program elements can beembodied by hardware and/or software (this includes firmware, residentsoftware, micro-code, etc.). Within the framework of the invention,computer program elements can take the fonii of a computer programproduct which can be embodied by a computer-usable, for examplecomputer-readable data storage medium comprising computer-usable, forexample computer-readable program instructions, “code” or a “computerprogram” embodied in said data storage medium for use on or inconnection with the instruction-executing system. Such a system can be acomputer; a computer can be a data processing device comprising meansfor executing the computer program elements and/or the program inaccordance with the invention, for example a data processing devicecomprising a digital processor (central processing unit or CPU) whichexecutes the computer program elements, and optionally a volatile memory(for example a random access memory or RAM) for storing data used forand/or produced by executing the computer program elements. Within theframework of the present invention, a computer-usable, for examplecomputer-readable data storage medium can be any data storage mediumwhich can include, store, communicate, propagate or transport theprogram for use on or in connection with the instruction-executingsystem, apparatus or device. The computer-usable, for examplecomputer-readable data storage medium can for example be, but is notlimited to, an electronic, magnetic, optical, electromagnetic, infraredor semiconductor system, apparatus or device or a medium of propagationsuch as for example the Internet. The computer-usable orcomputer-readable data storage medium could even for example be paper oranother suitable medium onto which the program is printed, since theprogram could be electronically captured, for example by opticallyscanning the paper or other suitable medium, and then compiled,interpreted or otherwise processed in a suitable manner. The datastorage medium is preferably a non-volatile data storage medium. Thecomputer program product and any software and/or hardware described hereform the various means for performing the functions of the invention inthe example embodiments. The computer and/or data processing device canfor example include a guidance information device which includes meansfor outputting guidance information. The guidance information can beoutputted, for example to a user, visually by a visual indicating means(for example, a monitor and/or a lamp) and/or acoustically by anacoustic indicating means (for example, a loudspeaker and/or a digitalspeech output device) and/or tactilely by a tactile indicating means(for example, a vibrating element or a vibration element incorporatedinto an instrument). For the purpose of this document, a computer is atechnical computer which for example comprises technical, for exampletangible components, for example mechanical and/or electroniccomponents. Any device mentioned as such in this document is a technicaland for example tangible device.

It is the function of a marker to be detected by a marker detectiondevice (for example, a camera or an ultrasound receiver or analyticaldevices such as CT or MRI devices) in such a way that its spatialposition (i.e. its spatial location and/or alignment) can beascertained. The detection device is for example part of a navigationsystem. The markers can be active markers. An active marker can forexample emit electromagnetic radiation and/or waves which can be in theinfrared, visible and/or ultraviolet spectral range. A marker can alsohowever be passive, i.e. can for example reflect electromagneticradiation in the infrared, visible and/or ultraviolet spectral range orcan block x-ray radiation. To this end, the marker can be provided witha surface which has corresponding reflective properties or can be madeof metal in order to block the x-ray radiation. It is also possible fora marker to reflect and/or emit electromagnetic radiation and/or wavesin the radio frequency range or at ultrasound wavelengths. A markerpreferably has a spherical and/or spheroid shape and can therefore bereferred to as a marker sphere; markers can however also exhibit acornered, for example cubic, shape.

A marker device can for example be a reference star or a pointer or asingle marker or a plurality of (individual) markers which are thenpreferably in a predetermined spatial relationship. A marker devicecomprises one, two, three or more markers, wherein two or more suchmarkers are in a predetermined spatial relationship. This predeterminedspatial relationship is for example known to a navigation system and isfor example stored in a computer of the navigation system.

In another embodiment, a marker device comprises an optical pattern, forexample on a two-dimensional surface. The optical pattern might comprisea plurality of geometric shapes like circles, rectangles and/ortriangles. The optical pattern can be identified in an image captured bya camera, and the position of the marker device relative to the cameracan be determined from the size of the pattern in the image, theorientation of the pattern in the image and the distortion of thepattern in the image. This allows to determine the relative position inup to three rotational dimensions and up to three translationaldimensions from a single two-dimensional image.

A navigation system, such as a surgical or medical navigation system, isunderstood to mean a system which can comprise: at least one markerdevice; a transmitter which emits electromagnetic waves and/or radiationand/or ultrasound waves; a receiver which receives electromagnetic wavesand/or radiation and/or ultrasound waves; and an electronic dataprocessing device which is connected to the receiver and/or thetransmitter, wherein the data processing device (for example, acomputer) for example comprises a processor (CPU) and a working memoryand advantageously an indicating device for issuing an indication signal(for example, a visual indicating device such as a monitor and/or anaudio indicating device such as a loudspeaker and/or a tactileindicating device such as a vibrator) and a permanent data memory,wherein the data processing device processes navigation data forwardedto it by the receiver and can advantageously output guidance informationto a user via the indicating device. The navigation data can be storedin the permanent data memory and for example compared with data storedin said memory beforehand.

A landmark is a defined element of an anatomical body part which isalways identical or recurs with a high degree of similarity in the sameanatomical body part of multiple patients. Typical landmarks are forexample the epicondyles of a femoral bone or the tips of the transverseprocesses and/or dorsal process of a vertebra. The points (main pointsor auxiliary points) can represent such landmarks. A landmark which lieson (for example on the surface of) a characteristic anatomical structureof the body part can also represent said structure. The landmark canrepresent the anatomical structure as a whole or only a point or part ofit. A landmark can also for example lie on the anatomical structure,which is for example a prominent structure. An example of such ananatomical structure is the posterior aspect of the iliac crest. Anotherexample of a landmark is one defined by the rim of the acetabulum, forinstance by the centre of said rim. In another example, a landmarkrepresents the bottom or deepest point of an acetabulum, which isderived from a multitude of detection points. Thus, one landmark can forexample represent a multitude of detection points. As mentioned above, alandmark can represent an anatomical characteristic which is defined onthe basis of a characteristic structure of the body part. Additionally,a landmark can also represent an anatomical characteristic defined by arelative movement of two body parts, such as the rotational centre ofthe femur when moved relative to the acetabulum.

Preferably, an atlas describes (for example defines, more particularlyrepresents and/or is) a general three-dimensional shape of an anatomicalbody part. The atlas therefore represents an atlas of the anatomicalbody part. An atlas typically consists of a plurality of generic modelsof objects, wherein the generic models of the objects together form acomplex structure. For example, the atlas constitutes a statisticalmodel of a patient's body (for example, a part of the body) which hasbeen generated from anatomic information gathered from a plurality ofhuman bodies, for example from medical image data containing images ofsuch human bodies. In principle, the atlas therefore represents theresult of a statistical analysis of such medical image data for aplurality of human bodies. This result can be output as an image—theatlas therefore contains or is comparable to medical image data. Such acomparison can be carried out for example by applying an image fusionalgorithm which conducts an image fusion between the atlas data and themedical image data. The result of the comparison can be a measure ofsimilarity between the atlas data and the medical image data.

The human bodies, the anatomy of which serves as an input for generatingthe atlas, advantageously share a common feature such as at least one ofgender, age, ethnicity, body measurements (e.g. size and/or mass) andpathologic state. The anatomic information describes for example theanatomy of the human bodies and is extracted for example from medicalimage information about the human bodies. The atlas of a femur, forexample, can comprise the head, the neck, the body, the greatertrochanter, the lesser trochanter and the lower extremity as objectswhich together make up the complete structure.

Image fusion can be elastic image fusion or rigid image fusion. In thecase of rigid image fusion, the relative position between the pixels ofa 2D image and/or voxels of a 3D image is fixed, while in the case ofelastic image fusion, the relative positions are allowed to change.

In this application, the term “image morphing” is also used as analternative to the term “elastic image fusion”, but with the samemeaning.

Elastic fusion transformations (for example, elastic image fusiontransformations) are for example designed to enable a seamlesstransition from one dataset (for example a first dataset such as forexample a first image) to another dataset (for example a second datasetsuch as for example a second image). The transformation is for exampledesigned such that one of the first and second datasets (images) isdeformed, for example in such a way that corresponding structures (forexample, corresponding image elements) are arranged at the same positionas in the other of the first and second images. The deformed(transformed) image which is transformed from one of the first andsecond images is for example as similar as possible to the other of thefirst and second images. Preferably, (numerical) optimization algorithmsare applied in order to find the transformation which results in anoptimum degree of similarity. The degree of similarity is preferablymeasured by way of a measure of similarity (also referred to in thefollowing as a “similarity measure”). The parameters of the optimizationalgorithm are for example vectors of a deformation field. These vectorsare determined by the optimization algorithm in such a way as to resultin an optimum degree of similarity. Thus, the optimum degree ofsimilarity represents a condition, for example a constraint, for theoptimization algorithm. The bases of the vectors lie for example atvoxel positions of one of the first and second images which is to betransformed, and the tips of the vectors lie at the corresponding voxelpositions in the transformed image. A plurality of these vectors ispreferably provided, for instance more than twenty or a hundred or athousand or ten thousand, etc. Preferably, there are (other) constraintson the transformation (deformation), for example in order to avoidpathological deformations (for instance, all the voxels being shifted tothe same position by the transformation). These constraints include forexample the constraint that the transformation is regular, which forexample means that a Jacobian determinant calculated from a matrix ofthe deformation field (for example, the vector field) is larger thanzero, and also the constraint that the transformed (deformed) image isnot self-intersecting and for example that the transformed (deformed)image does not comprise faults and/or ruptures. The constraints includefor example the constraint that if a regular grid is transformedsimultaneously with the image and in a corresponding manner, the grid isnot allowed to interfold at any of its locations. The optimising problemis for example solved iteratively, for example by means of anoptimization algorithm which is for example a first-order optimizationalgorithm, such as a gradient descent algorithm. Other examples ofoptimization algorithms include optimization algorithms which do not usederivations, such as the downhill simplex algorithm, or algorithms whichuse higher-order derivatives such as Newton-like algorithms. Theoptimization algorithm preferably performs a local optimization. Ifthere is a plurality of local optima, global algorithms such assimulated annealing or generic algorithms can be used. In the case oflinear optimization problems, the simplex method can for instance beused.

In the steps of the optimization algorithms, the voxels are for exampleshifted by a magnitude in a direction such that the degree of similarityis increased. This magnitude is preferably less than a predefined limit,for instance less than one tenth or one hundredth or one thousandth ofthe diameter of the image, and for example about equal to or less thanthe distance between neighbouring voxels. Large deformations can beimplemented, for example due to a high number of (iteration) steps.

The determined elastic fusion transformation can for example be used todetermine a degree of similarity (or similarity measure, see above)between the first and second datasets (first and second images). To thisend, the deviation between the elastic fusion transformation and anidentity transformation is determined. The degree of deviation can forinstance be calculated by determining the difference between thedeterminant of the elastic fusion transformation and the identitytransformation. The higher the deviation, the lower the similarity,hence the degree of deviation can be used to determine a measure ofsimilarity.

A measure of similarity can for example be determined on the basis of adetermined correlation between the first and second datasets.

In particular, the invention does not involve or in particular compriseor encompass an invasive step which would represent a substantialphysical interference with the body requiring professional medicalexpertise to be carried out and entailing a substantial health risk evenwhen carried out with the required professional care and expertise. Forexample, the invention does not comprise a step of positioning a medicalimplant in order to fasten it to an anatomical structure or a step offastening the medical implant to the anatomical structure or a step ofpreparing the anatomical structure for having the medical implantfastened to it. More particularly, the invention does not involve or inparticular comprise or encompass any surgical or therapeutic activity.The invention is instead directed as applicable to positioning a toolrelative to the medical implant, which may be outside the patient'sbody. For this reason alone, no surgical or therapeutic activity and inparticular no surgical or therapeutic step is necessitated or implied bycarrying out the invention.

The invention can be used for cranial, ENT, spinal and orthopedicimage-guided surgery products, in particular those of the applicant. Thez-Touch® laser registration system of the cranial and ENT navigationsystem of the applicant is considered as a unique registration method inthe neurosurgery domain because no dedicated navigation scan has to beperfotined and the existing optical tracking system can be used torecord the surface points. Still, some users find the registrationprocedure cumbersome because it is pretty hard for the software toconvey information to the user on where to scan the surface in order tooptimize the surface matching result. By automatically sampling thesepoints with the help of a robotic system, the user is not only relievedfrom this activity but also improved and more consistent registrationresults can be obtained by having the robot acquire points until asatisfactory result is achieved. By having the robotic system handle thez-Touch® acquisition device, optimal attributes regarding movementspeed, distance to surface and angulation towards the tracking systemcan be ensured which are tough to guarantee when handling the devicemanually. The doctor or technician is thus relieved from an activitywhich is often considered to be the most complex and error prone step inthe use of a surgical navigation system. This invention is especiallyrelevant if the robotic system is also used for subsequent steps of thesurgery because in this context there is no extra effort involved forhaving to put the robotic system in place. By maximizing the usagespectrum of a robotic system, the overall investment in such a systemcan be better justified.

BRIEF DESCRIPTION OF DRAWINGS

In the following, the invention is described with reference to theenclosed figures which represent preferred embodiments of the invention.The scope of the invention is not however limited to the specificfeatures disclosed in the figures, which show:

FIG. 1 a scenario with a patient to be registered;

FIG. 2 a workflow for registering the patient; and

FIG. 3 a registration system.

DETAILED DESCRIPTION

FIG. 1 shows a scenario with a patient P. A 3D image of the patient P isto be registered with the patient P. Registering the 3D image of thepatient with the patient P in this exemplary embodiment means to modifythe 3D image of the patient P, wherein the 3D image represents at leasta part of the surface of the patient P, with the actual patient P, thatis the surface of the patient P. In a registration based on a rigidfusion, only the six parameters of a virtual position of the 3D imageare determined such that the 3D image matches the patient P as good aspossible. In a registration based on an elastic fusion, an additionalmodification of the shape of the 3D image is performed.

In the present exemplary embodiment, the patient P rests on an operatingroom table 4 which carries a reference marker R. The reference marker Rdefines a reference co-ordinate system in which spatial locations ofpoints on the surface of the patient P are sampled and in which the 3Dimage of the patient P is aligned with the sampled surface points, andis thus registered with the patient P.

In the scenario shown in FIG. 1, a robot 1 is attached to the OR table4. The robot holds a surface point sampling device 2, which in turnbears a marker M rigidly attached thereto. A stereoscopic camera 3 of amedical tracking system images the marker M and the reference marker Rand determines the positions of those markers in a reference systemassociated with the camera 3. The position of the marker M in thereference system of the camera 3 can then be transformed into a positionin the reference system associated with the reference marker R.

It shall be noted that other constellations are possible. Theregistration of the patient P with a 3D image of the patient P can alsobe performed in the reference system of the camera 3. In this case, thereference marker R can be omitted. However, a marker can be attached tothe patient P such that a movement of the patient P relative to thecamera 3 can be detected and compensated when sampling the surfacepoints. An additional marker attached to the patient can also be used inthe configuration as shown in FIG. 1.

In the present exemplary embodiment, the surface point sampling device 2is of the contactless type. It basically comprises a laser beam sourceand generates a laser beam with a known orientation relative to themarker M. When the laser beam hits the patient P, it generates a laserspot on the surface of the patient P. The stereoscopic camera images thelaser spot and calculates the spatial location of the laser spot in itsown reference system. This spatial location is then transformed into aspatial location in the co-ordinate system of the reference marker R.

It shall be noted that the surface point sampling device can also be ofthe contact type. Such a surface point sampling device comprises acontact surface, such as a tip, which is to contact the surface of thepatient P. The location of the contact surface of the surface pointsampling device 2 relative to the marker M is known, such that thelocation of the contact surface, and therefore the sampled point on thesurface of the patient P, can be determined from the position of themarker M. The contact between the contact surface and the surface of thepatient P can for example be determined automatically, for example usinga pressure sensor in the surface point sampling device or the robot 1,or be indicated manually by a user.

FIG. 2 shows a flowchart of a workflow for registering the patient P.The workflow starts with the optional step S01 of defining one or moresafe working zones of the robot 1. A safe working zone is a spatial areain which the robot 1 can move without colliding with an object or aperson.

The workflow then proceeds to step S02 of acquiring a 3D image of thepatient P. The 3D image of the patient P represents at least a part ofthe surface of the patient P.

The workflow then proceeds to step S03 of performing a pre-registrationof the 3D image of the patient P with the patient P. Any suitabletechnique may be employed for performing the pre-registration, such asmatching certain points, such as landmarks identified in the 3D imageand a stereoscopic image of the patient P captured by the camera 3. Inanother approach, the pre-registration is based on a fusion of the 3Dimage of the patient P to a set of sampled points on the surface of thepatient P. Those sampled surface points can be sampled by use of therobot 1 or manually by a user who operates a surface point samplingdevice, such as the surface point sampling device 2, which can beattached to the robot 1 afterwards.

The pre-registration of step S03 is then used as an initial registrationfor the first iteration of the workflow.

The workflow then proceeds to step S04 of defining points on the surfaceof the patient P to be sampled. Due to the pre-registration of step S03,the approximate topography of the surface of the patient P is known, forexample in the reference system of the reference marker R or of thecamera 3. The approximate topography of the surface of the patient P canbe used to plan the surface points to be sampled. Criteria for definingthe points to be sampled can be at least one of maximizing the distancesto previously sampled surface points, selecting a point at a locationthat has a high curvature of the surface or is a saddle point of thesurface.

In addition, a point can be selected which is unobstructed by objectssuch as draping, tubing etc.

In addition or as an alternative, an optional step S03 a between stepsS03 and S04 involves registering an atlas with the patient P andextracting the points to be sampled from the registered atlas.

S04 might involve trajectory planning, which means to plan, and inparticular optimize, the order in which the surface points are to besampled. In one example, each surface point to be sampled is assigned acorresponding sampling position of the surface point sampling device 2,and the planned trajectory represents an order of the samplingpositions.

The workflow proceeds from step S04 to step S05 of sampling surfacepoints on the surface of the patient P according to the points to besampled as defined in step S04.

In step S05, the robot 1 subsequently moves the surface point samplingdevice 2 into a plurality of sampling positions, each corresponding to asurface point to be sampled. Once the surface point sampling device 2 isin a sampling position, the corresponding surface point is sampled. Itshall be noted that it is not necessary that the robot 1 stops thesurface point sampling device 2 at the sampling position. It is forexample sufficient that the speed of the surface point sampling device 2at the sampling position is below a predetermined threshold.

The workflow then proceeds to step S06 of updating the previousregistration according to the surface points sampled in step S05.

The workflow then proceeds to step S07 of determining whether or not theworkflow is in the first iteration. If this is the case, the workflowreturns to step S04. This means that steps S04, S05 and S06 areperformed at least twice. If the workflow is not in the first iteration,the workflow proceeds to step S08 of determining whether or not an exitcondition is fulfilled. If this is not the case, the workflow returns tostep S04. If this is the case, the workflow ends at step S09.

The exit condition can be at least one of a time span used for samplingsurface points, the total number of sampled surface points, the accuracyof the registration or the registration confidence.

A plurality of different approaches might be implemented for updatingthe registration in step S06. In one implementation, a completely newregistration is calculated based on the spatial locations of all sampledpoints which were sampled in all iterations of the workflow. In anotherimplementation, the registration after the previous iteration (or theinitial registration) is amended based on the spatial locations of thesampled points which were sampled in the current iteration of theworkflow.

It shall be noted that the robot is not necessarily attached to theoperation room table 4, but could also be attached to a wall or theceiling of the operating room or for example be attached to a fixed ormoveable base.

FIG. 3 shows a medical registration system 5 for obtaining aregistration of a 3D image of the patient P with the patient P. Themedical registration system 5 comprises the robot 1 holding the surfacepoint sampling device 2, the stereoscopic camera 3 and a computer 6.

The computer 6 comprises a central processing unit 7, an interface 8 anda memory 9. The memory 9 stores working data, such as spatial locationsof sampled points, the 3D image and the registration. It further storesinstructions which let the central processing unit 7 implement themethod or the workflow described herein.

The computer 6 is connected to the stereoscopic camera 3, an inputdevice 10, such as a mouse, a keyboard or a touch sensitive surface, andto an output device 11, such as a display or monitor. The connectionbetween the computer 6 and at least one of the camera 3, the inputdevice 10 or the output device 11 is implemented via the interface 8.

1.-15. (canceled)
 16. A method of operating a medical registrationsystem, the medical registration system comprising a robot which carriesa surface point sampling device and including a computer connected tothe robot, to obtain a registration of a 3D image of a patient with thepatient by: acquiring the 3D image of the patient; acquiring an initialregistration of the 3D image with the patient as the registration;sampling by the robot the spatial locations of N different points on thesurface of the patient using the surface point sampling device, whereinN is a positive integer; updating the registration based on the spatiallocations of the N sampled points; and repeating the sampling and theupdating steps until a predetermined exit condition is fulfilled. 17.The method of claim 16, wherein the predetermined exit condition is atleast one of a time lapse, the total number of sampled surface points,the accuracy of the registration or the registration confidence.
 18. Themethod of claim 16, wherein the sampling by the robot includes for eachof the N different points, instructing the robot to move the surfacepoint sampling device into a sampling position and instructing thesurface point sampling device to sample a surface point.
 19. The methodof claim 18, wherein sampling a surface point involves determining thedistance from the sampled surface point to the surface point samplingdevice and calculating the spatial location of the surface point fromsaid distance and the sampling position of the surface point samplingdevice.
 20. The method of claim 18, wherein the sampling position is aposition in which at least one of the distance or the orientationbetween the surface point sampling device and the surface of the patientis within a predetermined range.
 21. The method of claim 18, wherein thesampling position is a position in which a marker device attached to therobot or the surface point sampling device is within a predeterminedrange relative to a medical tracking system.
 22. The method of claim 16,wherein sampling a surface point involves moving, by the robot, thesurface point sampling device towards the surface of the patient untilit is in contact therewith.
 23. The method of claim 16, wherein asurface point to be sampled is determined based on the registered 3Dimage or a registered atlas.
 24. The method of claim 23, wherein asurface point to be sampled is determined based on a registered atlasand the atlas defines the number of points to be sampled in a predefinedarea of the surface.
 25. The method of claim 16, wherein a surface pointto be sampled is a point in an area of the surface with a high curvatureor comprising a saddle point.
 26. The method of claim 16, wherein asurface point to be sampled is a point with a maximum distance to theother surface points sampled so far.
 27. The method of claim 16, whereina surface point to be sampled is determined based on a co-registeredoptical image of the patient.
 28. The method of claim 16, wherein thesurface points to be sampled are selected such that the robot remains inone or more safe working areas.
 29. A non-transitory computer readablestorage medium having instructions for operating a medical registrationsystem, the medical registration system including a robot which carriesa surface point sampling device and a computer having at least oneprocessor connected to the robot, to obtain a registration of a 3D imageof a patient with the patient, the instructions comprising: acquiringthe 3D image of the patient; acquiring an initial registration of the 3Dimage with the patient as the registration; instructing the robot tosample the spatial locations of N different points on the surface of thepatient using the surface point sampling device, wherein N is a positiveinteger; updating the registration based on the spatial locations of theN sampled points; and repeating the instructing the robot to sample thespatial locations and the updating steps until a predetermined exitcondition is fulfilled.
 30. A medical registration system comprising acomputer having at least one processor, the computer connected to arobot which carries a surface point sampling device to obtain aregistration of a 3D image of a patient with the patient, the computerhaving instructions to perform the steps comprising: acquiring the 3Dimage of the patient; acquiring an initial registration of the 3D imagewith the patient as the registration; sampling by the robot the spatiallocations of N different points on the surface of the patient using thesurface point sampling device, wherein N is a positive integer; updatingthe registration based on the spatial locations of the N sampled points;and repeating the sampling and the updating steps until a predeterminedexit condition is fulfilled.